Partitioning and modularity of graphs with arbitrary degree distribution 
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We solve the graph bi-partitioning problem in dense graphs with arbitrary degree distribution using 
the replica method. We find the cut-size to scale universally with (y/k). In contrast, earlier results 
studying the problem in graphs with a Poissonian degree distribution had found a scaling with 
yj (k) [Fu and Anderson, J. Phys. A: Math. Gen. 19, 1986]. The new results also generalize 
to the problem of q-partitioning. They can be used to find the expected modularity Q [Newman 
and Grivan, Phys. Rev. E, 69, 2004] of random graphs and allow for the assessment of statistical 
significance of the output of community detection algorithms. 
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Given a graph or network Q(N,M) of N nodes and 
M edges, the problem of graph partitioning is finding 
a partition of the nodes into q equal sized parts, such 
that the number of edges connecting different parts, the 
cut-size, is minimal. The solution of this problem has 
many important practical applications in multiprocessor 
scheduling for parallel computing, VLSI chip design and 
data mining 

Due to the problem being NP-complete Q , only heuris- 
tic algorithms exist to approximate a solution. Further- 
more, one cannot determine in polynomial time, whether 
a solution found is indeed optimal. The only way to as- 
sess the quality of a heuristic algorithm or a particular 
result is to compare with rigorous bounds on the cut-size. 

In the past, such bounds have been established for 



Erdos-Renyi (ER) random 
sonian degree distribution 
Lattices with fixed valence 




exhibiting Pois- 
101 or Bethe- 
Ha |. only. All 



of these bounds express the expected cut-size in terms of 
the mean degree of the network. However, this is inap- 
propriate for a large number of problem instances which 
show fat tailed degree distributions 1(| 17 1 and we will 
show that the traditional bounds are inappropriate in 
these cases. 

Recently, the problem of graph partitioning has re- 
ceived considerable interest again in a more relaxed ver- 
sion, known as community detection or graph clustering 
which also has a broad range of applications for data 
analysis in the life sciences [3, [3] to social science [2(| ■ 
Given a graph the problem is now finding a partition 
into q arbitrarily sized communities or clusters which are 
densely connected internally, but only sparsely connected 
among each other [2l|, [23]. The hypothesis is that in 
many real world networks, nodes of differing types exist 
and that nodes of the same type have a higher proba- 
bility of being linked than nodes of different types [23| . 



Community detection thus aims at discovering precisely 
these latent classes of nodes. 

A widely used quality measure for the partition into 
communities is the so-called "modularity" Q defined by 
Newman and Girvan [24| . For a partition into q commu- 
nities of arbitrary size it is defined as Q q = ^2 q s=1 (e ss — 
a,g), with a s = J2l=i e rs- Here, e rs is the fraction of edges 
falling between nodes in groups r and s. Obviously, the 
larger the distortion between internal edges e ss and ex- 
pected internal edges a? s , i.e. the more modular a network 
is, the higher Q. By design, Q < 1. 

In case of q equal sized parts, we have a relation be- 
tween the cut-size C q of the partition and its modularity: 



C a = M 



1 



q 



(i) 



'Present Address: Institute for Theoretical Physics, University of 
Wiirzburg, Am Hubland, 97074 Wiirzburg, Germany 



We see that Q q measures the improvement of the parti- 
tion over a random assignment of nodes into parts. 

One problem of community detection and all cluster- 
ing techniques in general is that it is not clear a priori, 
whether latent classes of nodes in a particular network 
do exist. Numerical experiments have shown that ran- 
dom networks can be partitioned in a way which has a 
very high value of Q, but this does of course not imply 
that latent classes of nodes in the network 25, 26| in fact 
exist. One way to escape this "deception of randomness" 
[27| is by comparing the value of Q obtained by cluster- 
ing the empirical data to an expectation value for Q for 
an appropriate random null model. 

The partition of a purely random graph with maximum 
modularity is an equi-partition [261 ]. Therefore, solving 
the problem of graph partitioning for graphs with arbi- 
trary degree distribution, we can also give expectation 
values for the maximum attainable modularity in purely 
random graphs and vice versa. These expectation values 
can be used in the assessment of statistical significance 
of empirical data analysis. 

The problem of maximizing the modularity is equiva- 
lent to finding the ground state of a spin glass 
The corresponding Hamiltonian H for q = 2 relates to the 



2 



modularity Q2 at 7 = 1 via 



(k)NQ 2 =-H = J2( A v - lPij) S i S 3 



(2) 



where p^ — kikj/(k)N with (k) as the average degree 
in the network and Ay denotes the adjacency matrix of 
the graph such that J^ . Ay = fe,-, the degree of node 
i. Further, we use Ising spins Si — ±1. Following Ref. 
[30l | may be interpreted as the connection probability 
in the network due to J2i<jPij = M = J2i<j ^iji even 
though it is not strictly ensured that py < 1 for all 
The ground state energy of $2$ can be written as 



E„ 




E*<-£*< 



7(fc 2 ) 
2(fc) 



(3) 



Assuming equal sized parts with the same degree distri- 
bution and (k 2 ) being finite, the last two terms in ((3]) 
vanish and we see that (JTJ holds. Due to the existence 
of (A; 2 ), we can also assume the form of |>y to be a good 
approximation for the true probability distribution that 
parameterizes Aij. 

The Hamiltonian {2J) represents an infinite range spin 
glass with couplings Jy = (Ay — 7Py ) between all pairs 
of nodes. The mean of the coupling matrix is zero by 
construction for 7 = 1. Interestingly, we are dealing with 
a system that shows prop erties of both the Sherrington- 
Kirkpatrick (SK) 0,111] and the Viana-Bray (VB) type 
[33)]. The distribution of couplings is not symmetric. 
There are many weak anti-ferromagnetic couplings J~ 
between unconnected nodes and few but strong ferro- 
magnetic couplings J + between connected nodes: 

J?j = (1 - JPij), Jij = -7Pii, J?j - Jij = J (4) 



The parameter 7 allows tuning the strength of the anti- 
ferromagnetic interactions and comparing results for dif- 
ferent 7 enables us to detect hierarchies and overlap in 
community structures 0, [2^|. Note that J — 1 inde- 
pendent of the choice of the parameter 7. The individual 
steps of the following development follow closely along 
the lines of Fu and Anderson (FA) [jl, th ough a treatment 
along the lines of VB or Monasson [34[ is also possible. 

We use the replica trick 3l|, l35| in order to evaluate 



the logarithm of the partition function using [In Z] = 
lim„_>o([^"] — I)/"- The symbol [•] denotes an expec- 
tation value over the ensemble of graphs with a given 
degree distribution. The probability density function of 
the coupling matrix Jy is then defined as 



Qij(Jij) = Pi 3 5{Ji 3 - Jtj) + (1 - Pij) s ( J ij ~ J ij)- ( 5 ) 

The here is the same as that in equations ([2]) and 
The mean of this distribution is p(l — 7) and we are 
dealing with a diluted ferromagnet for 7 < 1 and with a 
diluted anti- ferromagnet 7 > 1. For all 7 > 1 we hence 
expect zero magnetization in the ground state. 

With these preliminaries, we can now follow the steps 
of FA [5( and average the replicated partition function 
over the ensemble of graphs with given degree distribu- 
tion: 



[Z n ] = J n^^(^)Tr n exp J ^^W? 

i,j [ a i<j 

(6) 

Each of the integrals can be performed separately and 
after some manipulation we arrive at 



\Z n \ = Tr n exp 



E 

i<j 



0(jr j+Pij j)J2s?S? 



-p^ 



(7) 



which is still in complete correspondence to Ref. Q. Now we deviate from FA's development. Instead of a constant 
Pij = p, we use the factorized form introduced earlier. Neglecting terms involving pf^ over those involving py , dropping 
all terms which vanish for large N or for n — > and rearranging the sums we arrive at: 



[Z n ] = exp ^- — j — J lv u exp 



(k)N 



(8) 



Note that we would have arrived at the same expres- sion by using a Gaussian coupling distribution for qij(Jij) 



3 



with mean (J— 7) /(k)N > and variance cr?- = kikj/(k)N. 
At this point, we have a form that is in principle equiva- 
lent to the SK model except for the fc^ in the sums over 
spins. We stress again that setting 7 = 1 leads to vanish- 
ing magnetization and thus corresponds to a bi-partition. 
The further development now proceeds along the lines of 
SK and we arrive at one single site problem for each de- 
gree. 

Finally we can derive the replica symmetric equations 
of state similar to Ref. 36]: 

m* = y|y^>(fc)fc J Dzta,nh(f3H k (z)), (9) 
q* = 7^y5>(fc)fc J Dztanh 2 (j3H k (z)). (10) 



The Gaussian measure Dz = exp(— z /2)/v27r indicates 
that the local fields for the spins are also Gaussian dis- 
tributed, but slightly different for every degree. Nodes 
of the same degree see the same field distribution. The 
effective Hamiltonian is 



H k {z) = Jy 



7)fcm*. 



(11) 



We can now again follow standard procedures and find 
the zero temperature limit of the free energy and hence 
Q 2 : 



1 



77T lim [/] 

(fc) /3^<X) 



c/ J(fc 1/2 )/(fc) 



(12) 



where U is the absolute value of the ground state en- 
ergy per spin of the Sherrington-Kirkpatrick model. It 
is interesting to note the similarity with FA, who give 
Qi = Uq\J{\ — p)/Np from the ground state energy of 
the bi-partitioning problem on Erdos-Renyi (ER) graphs 
0, Q with link probability pij = p. For graphs with fixed 
degree, the two formulations of FA and (fT2|) are equiva- 
lent. The following numerical experiments will show the 
adequacy of (TT2"]) for a wide range of degree distributions. 

First ER networks of N = 10, 000 nodes and average 
degree between 3 and 20 were created. The second en- 
semble of networks is that of scale free networks with 
iV = 10, 000 nodes and a degree distribution of the form 
p(k) oc fc _K (SFkmin)- The maximum possible degree 
was set to 1000 and the graphs were composed using the 
Molloy-Reed algorithm [37[ . The second moment of the 
distribution exists for all k > 2. We chose n — 3 for 
our experiments. In order to produce graphs of different 
average degree, we introduced a minimum degree fc m m, 
such that p{k < k m i n ) — with 2 < fc m j n < 12. Finally, 
we study a second class of scale free networks (SF A k). 
Introducing a k m i n may have been a too drastic step, 
as it excludes all nodes of smaller degree from the net- 
work. Therefore, we modify the degree distribution to 
p(k) — (k + Afc) _K . Using k = 3 as before and varying 
Afc between 2 and 20. 
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FIG. 1: Numerical experiments maximizing the modularity of 
bi-partions in random graphs of different topologies (Erdos- 
Renyi random graphs and two forms of a scale free degree dis- 
tribution, see text for details. Scaling Q in units of (yk)/(k) 
collapses the data points onto one universal curve as expected 
from (|12[) . The dashed and solid lines correspond to Uo in the 
replica symmetric and replica symmetry breaking case. 



Bi-partitions were calculated by minimizing the Hamil- 
tonian |(3J) using simulated annealing. As expected, the 
ground state was found to have zero magnetization. Fig- 
ure[T]shows the result of this experiment. The empirically 
found maximum modularity for a bi-partition in networks 
from these three classes is rescaled in units of (\/k) / (k) 
and plotted against (fc) for better visibility. According 
to (TT2"]) we expect the curves to coincide on a horizontal 
line at Uq. Indeed, this rescaling collapses the values for 
different topologies together. The dashed line represents 
the ground state energy in the replica symmetric approx- 
imation Uq ,s = \/2 /n while the solid line corresponds to 
U* SB = 0.765 0, HI, the full RSB SK ground state 
energy. Note that our treatment of the problem allows 
for the same RSB calculations as the ordinary SK model 
as all fcj can be drawn out of sums over replica indices. 
Clearly, we see that (TT2")) gives a good approximation of 
the modularity and hence cut-size for the different topolo- 
gies. As expected from the mean field nature of our cal- 
culation, the estimate is better for denser graphs. 

In contrast, Figure [2] shows the same data points as 
Figure [1] but this time rescaled in units of -^/(l — p)/(k) 
as suggested by FA. From the fact that the data points 
do not collapse onto a single universal curve, we can con- 
clude the inadequacy of this scaling for graphs with a 
broad degree distribution. Note, that (TT2")) also improves 
over the results of FA in the case of ER graphs. 

Our results from the case of bi-partitioning generalize 
in a straightforward way to the case of q-partitioning and 
we can simply replace the scaling from the formulas by 
Kanter and Sompolinsky (KS) [6] or Lai and Goldschmidt 
Q. The expectation value for the maximum modularity 
of a random graph with arbitrary degree distribution is 
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FIG. 2: The same data points as in Figure[TJ but Q in units of 
\/(l — p)/Np as suggested by Fu and Anderson Clearly, 
the parametrization by mean degree alone cannot account 
for the observed cut sizes/modularities in the three different 
topologies. 



then 1261: 



Q = 0.97 



(Vk) 
(k) 



(13) 



The factor of Uq = 0.97 corresponds to the ground state 
energy of the Potts glass in the 1-RSB treatment as cal- 
culated by KS [f| . Figure [3] shows the maximum 
modularity obtained when minimizing the Hamiltonian 
<j2j> in the same graphs as used in Fig. [T] but with the 
number of communities q as a free parameter. Again, 
we see that plotting Q in units of (Vk) / (k) collapses the 
data points onto a single universal curve. The approx- 
imation of the universal value of Uq however, is much 
slower. This was already observed in Ref. [261 ] and can 
be explained by the fact that sparser graphs tend to clus- 
ter into more modules than predicted by KS. This leads 
to a higher number of degrees of freedom which then 
tend to accommodate better for fluctuations in the link 
structure of sparse graphs which lead to relatively higher 
modularities or lower cut-sizes than expected for denser 
graphs. Since the rescaling of the modularity has made 
the data points from all topologies collapse onto a single 
universal curve, we can employ a recursive estimate us- 
ing eq. (fT2"| several times as outlined in Ref. (29| . It is 
interesting to note that once we have collapsed the data 
onto a universal curve, we can use the recursive estimate 
from the convenient ER graphs to give analytical bounds 
for other topologies as well. 

In conclusion, we have presented new approximations 
for the cut-size of a bi-partition and the corresponding 
modularity in dense graphs with arbitrary degree distri- 
butions. Our results show that scale free networks ex- 
hibit larger cut-sizes and smaller modularities than ER 
graphs of the same average degree. Graphs with a modi- 
fied scale free degree distribution p(k) oc (k + Ak) K have 



1,4; 
1,2; 



~m 1! 

V 
A 

^0,8; 

a 



*3U 



0,6 h 

0,4^ 




+ ER 

x SF, . 

kmin 
* SF Ak 

- 1-RSB 

-- recursive bipart. 



10 



<k> 



15 



20 



25 



FIG. 3: Numerical experiments maximizing the modularity in 
random graphs of different topologies (Erdos-Renyi random 
graphs and two forms of a scale free degree distribution, see 
text for details. Scaling Q in units of (vk) / (k) collapses the 
data points onto one universal curve as expected from (I12|l . 
The solid and solid lines correspond to ground state energy 
of the Potts glass in 1-RSB approximation due to Kanter and 
Sompolinsky The dashed line corresponds to an estimate 
obtained from a recursive bi-paritioning along the lines of Ref. 
[2(| using equation 1 121 



larger cut-sizes and smaller modularities than those with 
a minimum degree. This result also gives new insight 
into the stability of networks to targeted attack (link re- 
moval) [391 ]. The estimation of modularity of random 
graphs presented allows for a convenient comparison of 
the output of community detection algorithms with ap- 
propriate random null models and hence gives a first clue 
on the statistical significance of the findings in real world 
networks and paves the way to understanding the theo- 
retical limits of community detection [ioj ]. 
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